Skip to content

✨ dynamic-scheduler prerequisites ⚠️🚨 #8287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Sep 4, 2025
Merged

✨ dynamic-scheduler prerequisites ⚠️🚨 #8287

merged 15 commits into from
Sep 4, 2025

Conversation

@GitHK
Copy link
Contributor

@GitHK GitHK commented Sep 2, 2025 •
edited
Loading

What do these changes do?

  • ⬆️ upgraded nicegui to latest version
  • 🎨 director-v2's scheduler can now be switched off (exposed env var in config)
  • 🎨 deferred tasks now allow to wait between retries by a specified amount of time
  • ✨ Deferred tasks now triggers on_cancelled event after the cancellation of the task was handled

Below a preview of the updated DeferredTasks diagram

stateDiagram-v2
    (1) --> Scheduled: via [start]
    (2) --> ManuallyCancelled: via [cancel]

    ManuallyCancelled --> Worker: attempts to cancel task in

    Scheduled --> SubmitTask
    SubmitTask --> Worker

    ErrorResult --> SubmitTask: try again
    Worker --> ErrorResult: upon error
    ErrorResult --> FinishedWithError: gives up when out of retries or if cancelled
    Worker --> DeferredResult: success

    DeferredResult --> (3): calls [on_result]
    FinishedWithError --> (4): calls [on_finished_with_error]
    Worker --> Removed*: task cancelled
    Removed* --> (5): calls [on_cancelled]
Loading

Related issue/s

How to test

Dev-ops ⚠️🚨

  • requires the following MR to be merged before merging this
  • Via redis_commander remove all entries inside the DEFERRED_TASKS database

@GitHK GitHK self-assigned this Sep 2, 2025
@GitHK GitHK added this to the Cheops milestone Sep 2, 2025
@codecov
Copy link

codecov bot commented Sep 2, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 92.50000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.84%. Comparing base (62244f5) to head (39c0c04).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #8287      +/-   ##
==========================================
- Coverage   87.85%   87.84%   -0.01%     
==========================================
  Files        1940     1940              
  Lines       75276    75306      +30     
  Branches     1309     1311       +2     
==========================================
+ Hits        66131    66153      +22     
- Misses       8752     8757       +5     
- Partials      393      396       +3
Flag Coverage Δ
integrationtests 64.08% <ø> (+<0.01%) ⬆️
unittests 86.51% <92.50%> (+<0.01%) ⬆️
Components Coverage Δ
pkg_aws_library 93.59% <ø> (ø)
pkg_celery_library 87.37% <ø> (ø)
pkg_dask_task_models_library 79.62% <ø> (ø)
pkg_models_library 93.13% <ø> (ø)
pkg_notifications_library 85.20% <ø> (ø)
pkg_postgres_database 88.02% <ø> (ø)
pkg_service_integration 70.19% <ø> (ø)
pkg_service_library 71.13% <92.30%> (+0.07%) ⬆️
pkg_settings_library 90.17% <ø> (ø)
pkg_simcore_sdk 84.85% <ø> (-0.18%) ⬇️
agent 93.53% <ø> (ø)
api_server 91.93% <ø> (ø)
autoscaling 95.77% <ø> (ø)
catalog 92.34% <ø> (ø)
clusters_keeper 99.13% <ø> (ø)
dask_sidecar 91.81% <ø> (-0.57%) ⬇️
datcore_adapter 97.94% <ø> (ø)
director 75.81% <ø> (-0.09%) ⬇️
director_v2 90.98% <ø> (-0.06%) ⬇️
dynamic_scheduler 96.27% <100.00%> (ø)
dynamic_sidecar 90.46% <ø> (ø)
efs_guardian 89.62% <ø> (ø)
invitations 91.44% <ø> (ø)
payments 92.61% <ø> (ø)
resource_usage_tracker 92.13% <ø> (ø)
storage 86.57% <ø> (+0.08%) ⬆️
webclient ∅ <ø> (∅)
webserver 88.03% <ø> (+0.02%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62244f5...39c0c04. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@GitHK GitHK changed the title ✨ Adding core scheduling capabilities to dynamic-scheduler ✨ Adding core scheduling capabilities to dynamic-scheduler ⚠️🚨 Sep 2, 2025
@mergify
Copy link
Contributor

mergify bot commented Sep 2, 2025 •
edited
Loading

🧪 CI Insights

Here's what we observed from your CI run for 39c0c04.

🟢 All jobs passed!

But CI Insights is watching 👀

@GitHK GitHK changed the title ✨ Adding core scheduling capabilities to dynamic-scheduler ⚠️🚨 ✨ dynamic-scheduler prerequisites ⚠️🚨 Sep 3, 2025
@GitHK GitHK marked this pull request as ready for review September 3, 2025 09:11
pcrespov
pcrespov approved these changes Sep 3, 2025
View reviewed changes
Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx

@sonarqubecloud
Copy link

sonarqubecloud bot commented Sep 4, 2025

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

@GitHK GitHK enabled auto-merge (squash) September 4, 2025 05:37
@GitHK GitHK merged commit c8f61c1 into ITISFoundation:master Sep 4, 2025
94 of 95 checks passed
@GitHK GitHK deleted the pr-osparc-migrate-dy-scheduler-part-1 branch September 4, 2025 05:55
@pcrespov pcrespov mentioned this pull request Sep 11, 2025
Closed
16 tasks
This was referenced Sep 18, 2025
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pcrespov pcrespov pcrespov approved these changes

@YuryHrytsuk YuryHrytsuk YuryHrytsuk approved these changes

@matusdrobuliak66 matusdrobuliak66 matusdrobuliak66 approved these changes

@sanderegg sanderegg Awaiting requested review from sanderegg sanderegg is a code owner

@mrnicegyu11 mrnicegyu11 Awaiting requested review from mrnicegyu11 mrnicegyu11 is a code owner

@giancarloromeo giancarloromeo Awaiting requested review from giancarloromeo

Assignees

@GitHK GitHK

Labels

None yet

Projects

None yet

Milestone

Cheops

Development

Successfully merging this pull request may close these issues.

4 participants

@GitHK @pcrespov @YuryHrytsuk @matusdrobuliak66